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ABSTRACT 

NASA’s Synthetic Vision Systems (SVS) project is developing technologies with practical applications to eliminate low 
visibility conditions as a causal factor to civil aircraft accidents while replicating the operational benefits of clear day 
flight operations, regardless of the actual outside visibility condition. A major thrust of the SVS project involves the 
development/demonstration of affordable, certifiable display configurations that provide intuitive out-the-window 
terrain and obstacle information with advanced pathway guidance for transport aircraft. The SVS concept being 
developed at NASA encompasses the integration of tactical and strategic Synthetic Vision Display Concepts (SVDC) 
with Runway Incursion Prevention System (RIPS) alerting and display concepts, real-time terrain database integrity 
monitoring equipment (DIME), and Enhanced Vision Systems (EVS) and/or improved Weather Radar for real-time 
object detection and database integrity monitoring. 

A flight test evaluation was jointly conducted (in July and August 2004) by NASA Langley Research Center and an 
industry partner team under NASA’s Aviation Safety and Security, Synthetic Vision System project. A Gulfstream G- 
V aircraft was flown over a 3 -week period in the Reno/Tahoe International Airport (NV) local area and an additional 3- 
week period in the Wallops Flight Facility (VA) local area to evaluate integrated Synthetic Vision System concepts. 
The enabling technologies (RIPS, EVS and DIME) were integrated into the larger SVS concept design. This paper 
presents experimental methods and the high level results of this flight test. 
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1. INTRODUCTION 

Limited visibility and reduced situation awareness have been cited as predominant causal factors for Controlled Flight 
Into Terrain (CFIT) accidents, where a functioning airplane is inadvertently flown into the ground, water, or an obstacle. 
In commercial aviation, over 30-percent of all fatal accidents worldwide are categorized as CFIT. 1 Another type of 
accident typically involving the same causal factors of restricted visibility and compromised situation awareness is 
runway incursions. Although the number of reported occurrences have decreased (339 in FY 2002 and 324 in FY 2003) 
from an all-time high in FY 2001 (407 occurrences), runway incursions are still a significant threat to aviation safety 
and operational efficiency. 2 Finally, the single largest factor causing airport delays is limited runway capacity and 
increased air traffic separation required when weather conditions fall below visual flight rule operations. 

1.1. NASA’s Synthetic Vision Systems Concept 

The Synthetic Vision Systems (SVS) project of the National Aeronautics and Space Administration’s (NASA) Aviation 
Safety and Security Program (AvSSP) is developing technologies with practical applications to eliminate low visibility 
conditions as a causal factor to civil aircraft accidents while replicating the operational benefits of clear day flight 
operations, regardless of the actual outside visibility condition. SVS emphasizes the cost-effective use of 
synthetic/enhanced- vision displays; worldwide navigation, terrain, obstruction, and airport databases; Global 
Positioning System (GPS)-derived navigation; and ranging and imaging sensors to eliminate “visibility-induced” (lack 
of visibility) errors for all aircraft categories. The SVS concept includes the intuitive display of intended flight path by 
tunnel or pathway-in-the-sky presentations. When coupled with a synthetic or enhanced view of the outside world, the 
spatially-integrated depiction of the intended aircraft flight path and its relation to the world provides an intuitive, easily 
interpretable display of flight-critical information for the pilot. The SVS concept being developed at NASA 
encompasses the integration of tactical and strategic Synthetic Vision Display Concepts (SVDC) with Runway 
Incursion Prevention System (RIPS) alerting and display concepts, real-time terrain database integrity monitoring 
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equipment (DIME), and Synthetic Vision Sensors (SV-Sensors), using an enhanced weather radar for real-time object 
detection, runway confirmation, and database integrity monitoring. 

1.2. SVS Flight Test Activities 

Previous flight tests of SVS have primarily focused on the general use and utility of SVS for providing flight critical 
guidance and improved terrain/situation awareness. 3,4 The research objectives of these previous flight tests also focused 
on SVS implementation issues, such as display requirements (e.g., size, content, and format) and on the development of 
SVS enabling technologies (e.g., RIPS, SV-Sensors, and DIME). While research to date has proven that precision 
navigation and on-board databases can provide the primary framework for substantial improvements in terrain/situation 
awareness with SVS, independent integrity monitors are envisioned as an integral component of a Synthetic Vision 
System to meet flight-critical safety requirements. This functionality is being developed by NASA and others to utilize 
existing on-board sensors (e.g., weather radars, high quality radar altimeters) to facilitate implementation. Specific on- 
board integrity functions include independent air-to-air, air-to-ground, ground-to-ground, and ground-to-air traffic and 
object/obstacle detection and surveillance, runway incursion monitoring, and database integrity and registration 
(navigational position confirmation via terrain feature extraction). Additionally, SVS concepts are being developed to 
augment and complement the independent capabilities of weather-penetrating, enhanced vision imaging sensors during 
low visibility landing and surface operations conditions. These technologies form the basis for monitoring the dynamic 
flight environment and thereby supplementing the synthetic world with real-time, direct measurement of the 
surrounding terrain and air/ground traffic for flight-critical applications. 

A flight test evaluation was jointly conducted (in July and August 2004) by NASA Langley Research Center and an 
industry team under NASA’s Aviation Safety and Security Program Synthetic Vision Systems project. A Gulfstream 
G-V aircraft was flown over a 3 -week period in the Reno/Tahoe International Airport (RNO) local area and an 
additional 3-week period in the Wallops Flight Facility (WAL) local area to evaluate an integrated Synthetic Vision 
System concept, including real-time, integrity monitoring functions. The Gulfstream G-V aircraft is owned, modified, 
and operated by Gulfstream Aerospace Corporation who performed this work as a subcontractor to Research Triangle 
Institute, working under a NASA contract. 

1.3. Flight Test Objectives 

Although past NASA flight tests has achieved significant progress in the development of both enabling technologies 
and synthetic vision display concepts, the research has demonstrated their efficacy as independent technologies. 
Therefore, the G-V Synthetic Vision Integrated Technology Evaluation (GVSITE) flight test was designed to evaluate 
an integration of the technologies critical to the development, and subsequent fielding, of actual Synthetic Vision (SV) 
Systems. In this context, SV systems include computer-generated terrain presented on Primary Flight Displays (PFD) 
and Electronic Attitude and Direction Indicators in place of the conventional blue sky and brown ground; monochrome 
textured terrain presented on Head-Up Displays (HUD); plan view or perspective views of computer generated terrain 
and obstacles on Navigation Displays (ND); and datalink, sensors, and algorithms to provide and verify required 
information for display. In addition, symbology and algorithms designed to enhance pilot situational awareness during 
surface and landing phase operations, and prevent or alert to potential runway incursions, was also part of the SV 
system tested during the GVSITE flight test. 

The primary GVSITE flight test objective was to evaluate in-flight an integrated tactical and strategic SVS concept 
intended for commercial and business aircraft in a terrain-challenged operational environment. Further, emphasis was 
placed on conducting operations at airports with crossing runways to support runway incursion prevention algorithm 
development. In addition to these NASA research objectives, Rockwell-Collins, Ohio University, and Rannoch 
Corporation participated as NASA Cooperative Research Agreement partners. The results of their work are reported 
elsewhere. 

Each technology area of an integrated Synthetic Vision System also carried specific development objectives as 
highlighted in the following sections. 
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1.3.1. SVDC Flight Test Objectives 

The SVDC provide the human-machine interface to the synthetic vision system. Display elements include, for example: 
perspective terrain, flight path guidance, and traffic information both in the air and on the surface. These display 
elements were presented on multiple display surfaces (HUD, PFD, and ND). 

The SVDC flight test objectives were as follows: 

1. Investigate operational utility and acceptability of enhanced terrain awareness of SVS display concepts to 
Required Navigation Performance (RNP) Approach-like procedures in a terrain-challenged operational 
environment. 

a. Assess pilot path control performance during manually-flown landing approach and go-around 
maneuvers in a terrain- challenged operational environment, with and without SVS display concepts, 
and determine the effect on that performance of the presence of SVS components. 

b. Assess pilot workload and situation awareness during manually- flown landing approach and go- 
around maneuvers in a terrain-challenged operational environment, with and without SVS display 
concepts. 

2. Assess symbology transition strategies on the head-up display, primary flight display and the navigation 
display for air to ground operations and ground to air operations. 

3. Evaluate hybrid terrain texturing concept (photorealistic with elevation-based color banding) on the ND 
with simulated Warning And Caution Overlays (WACO) driven from Terrain Awareness and Warning 
System (TAWS) inputs. 

4. Assess methods of presenting database loss-of- integrity alerts to the crew. 

5. Evaluate methods of iconic presentation of sensor-detected objects (unmapped towers, runway and taxiway 
obstacles, runway confirmation outline) from a weather radar and/or a forward-looking infrared (FLIR) 
camera into the SVS database presentation. 

6. Evaluate real-time insertion of iconic objects (e.g. towers, closed runway / taxiway) from a Notice to 
Airmen (NOTAM) source (using a simulated data link) into the SVS database presentation. 

1.3.2. RIPS Flight Test Objectives 

RIPS uses advanced displays, data links, and GPS to enable equipped aircraft to operate at airports unrestricted by 
visibility while ensuring safety. This is done by providing pilots with intuitive displays of their cleared taxi route, 
guidance cues, a real-time display of airport traffic, and alerts of runway incursions and route deviations on both a HUD 
and on head-down displays using an electronic moving map (EMM) of the airport. 

The RIPS objectives were as follows: 

1 . Evaluate RIPS integrated with the SVDC in an operational environment 

a. Assess whether runway incursion alerting can eliminate incursion-induced accidents during landing, 
takeoff, and taxi operations, emphasizing crossing runway scenarios. 

b. Evaluate situational awareness and guidance provided by RIPS to reduce the likelihood of inadvertent 
route deviation or ownship runway incursion due to blunder during taxi in any visibility condition. 

2. Validate proposed operational and technical requirements by comparing test data with draft standards and 
requirements. 

3. Assess technology performance through the collection and analysis of flight test data, assess the capability of 
the ADS-B data link, sensor object detection systems, and onboard incursion alerting system to support the 
system concept and operational requirements. 

1.3.3. SV-Sensors Flight Test Objectives 

The SV-Sensors “research sensor suite” included an advanced Weather Radar (WxR), a Forward-Looking Infrared 
System (FLIR), Automatic Dependent Surveillance-Broadcast (ADS-B) system, and Traffic Alert and Collision 
Avoidance System (TCAS) hardware and software, along with the software modules and interfaces that allowed for the 
evaluation of the integrated NASA SVS. 

The SV-Sensors objectives were as follows: 

1. Investigate the operational characteristics and compatibility of SV-Sensors technologies to provide terrain 
and obstacle information in air-to-air, air-to-ground, ground-to-ground, and ground-to-air flight phases. 



2. Assess the performance of SV-Sensors technologies and their ability to supplement the SVS concept. 

1.3.4. DIME Flight Test Objectives 

The DIME subsystem is designed to provide a quantifiable level of integrity for the Digital Elevation Models (DEMs) 
used by the NASA SVS. In general, this is accomplished by performing a real-time comparison of the DEM with 
measurements made by onboard ranging sensors. 

The DIME objectives were as follows: 

1. Characterize DIME operational behavior using the standard Gulfstream-V radio altimeter, a prototype 
advanced Weather Radar, the standard Inertial Navigation System (INS), and a GPS receiver configured to 
utilize the Wide-Area Augmentation System (WAAS). 

2. Assess the quality of multiple DEMs provided by Industry and Government organizations. 

3. Observe the integrity monitor performance in a region characterized by severe terrain undulations and in 
the presence of known and unknown DEM error classes (including intentional horizontal and vertical 
biases). 

2. METHODOLOGY 


2.1. Subjects 

Ten evaluation pilots (EPs), representing the airlines, a major transport aircraft manufacturer, the Federal Aviation 
Administration, and the Joint Aviation Authority, flew research flights totaling approximately 45 flight test hours. One 
hundred and forty-five flight test runs were conducted to evaluate the NASA SVS concepts at WAL (8 pilots) and RNO 
(7 pilots) airports. Five of the ten EPs flew at both test locations. All participants were HUD qualified. Prior to 
deployment, all EPs received a one-day training course (briefing and simulator session) at NASA Langley Research 
Center to familiarize them with the SVS concepts. Before the evaluation flights, the EPs received a “refresher” briefing 
summarizing the experiment objectives, pilot controls, display concepts, and experimental procedures. 

2.2. Test Aircraft 

The flight test was conduced using a Gulfstream G-V aircraft, owned, modified and operated by the Gulfstream 
Aerospace Corporation (Fig. 1). The left seat of the G-V was occupied by the EP and the right seat was occupied by a 
Gulfstream Safety Pilot (SP). The left seat equipage was modified to fit two Rockwell-Collins research displays for 
display of the SVS PFD and ND concepts, a new overhead HUD projection unit for evaluation of head-up concepts, and 
a voice recognition and speech (VRS) system for the pilot- vehicle interface to the SV displays (Fig. 1). A vision 
restriction device (VRD) was placed in the left-seat forward windscreen to block the EP’s forward vision and thus, 
simulate Instrument Meteorological Conditions (IMC) when needed experimentally. 



Figure 1. G-V aircraft exterior view and interior views with head-up and head-down displays. 

2.2.1. Head-Down Displays 

Two Rockwell-Collins 8”x8” Head-Down Displays (HDDs) were installed on the left hand side of the cockpit (Fig. 1) 
which could be driven directly from Personal Computers (PCs). The displays provided approximately 768x768 pixel 
resolution, driven from an XGA video input. The PCs generated all of the experimental Synthetic Vision Display 
Concepts. 
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2.2.2. Head-Up Display 

A modified Rockwell-Collins Flight Dynamics HGS-3300 HUD projection unit and combiner was installed in place of 
the normal G-V HUD projection unit and combiner. The HGS-3300 was modified to accept HGS-4000 computer 
inputs. The HGS-4000 HUD computer is a NASA research unit which allows stroke symbology programmability via 
an Arinc 429 communications bus. The HGS-4000 includes a stroke-on-raster capability. An RS-343 video input, 
driven from the research PCs, was used exclusively for this flight test to provide the best raster signal resolution. The 
total field-of-view of the HGS-3300 is 32° Horizontal (H) by 27° Vertical (V). 

Sunlight readability of SV HUD concepts was of particular interest in this test based upon previous flight test research 
results. 4 Synthetic vision HUD concepts must meet the same high ambient lighting conditions required of HUD 
symbology since the need for terrain awareness and the possibility of limited outside visibility are not restricted to low 
ambient lighting conditions only. To improve raster readability in high ambient lighting conditions, HUD combiner 
sunvisors were tested. The sunvisors were fabricated from four different shades of neutral density (gray) cast acrylic. 
The sunvisors were attached to the HUD combiner by Velcro to allow easy installation and removal. The sunvisors 
provide improved raster readability by increasing the contrast ratio of the display through ambient light attenuation. 

A HUD camera was designed and installed to directly record the HUD symbology and the out-the-window scene 
through the HUD combining glass. 

2.2.3. Voice Recognition and Speech System 

A VRS system was installed in the Gulfstream-V as a pilot- vehicle interface. The VRS was used primarily to facilitate 
the installation of a pilot- vehicle interface to the SVS displays without having to modify hardware or basic ship’s 
systems. It was also used as a way to test the use and utility of a VRS for future commercial and business aircraft flight 
deck developments. 

The bi-directional VRS allowed the EP to verbally command changes to the SVS displays and provided aural warnings 
and alerts to the crew when triggered by the SVS research systems. The VRS function was created using a Microsoft 
Windows-based application resident on a single computer housed in a 1U Rack Mount Case. The application used a 
commercial speech recognition engine to interpret the EP’s speech input, an interface application that passed and 
received information to/from an SGI 0-300 computer connected via Ethernet, and a synthesis (text-to-speech) module 
that generated aural messages or played pre-recorded “wave” files. The speech recognition product provided a 
commercial off-the-shelf, speaker-independent speech recognizer with easily tailored grammar. Rather than 
establishing a natural language environment, the speech recognition grammar was set-up as a hierarchy to improve 
recognition rates. The VRS used a 3 word command grammar to issue commands. The first word was the display 
device, the second word was the function and the third word was the value. For example, the command “NAV RANGE 
5” would set the navigational display range to 5 nautical miles. The pilot would press a button and speak a command 
which was interpreted by the VRS. The VRS was set such that if it was at least 40% confident in its interpretation, it 
would broadcast the command to the displays. The interpreted command was then momentarily displayed to the pilot 
for verification. 

2.2.4. Vision Restriction Device 

A means of simulating IMC flight for the EP was critical to meeting the flight test objectives given the predominance of 
Visual Flight Rules (VFR) conditions during the test period. The VRD was installed at the left crew station, forward of 
the HUD Combiner Glass. The VRD restricted the forward vision of the EP without compromising the safety pilot’s 
outside view. The EP’s side window view was not obstructed by this device. The VRD was designed to be quickly 
removed by the SP and was always removed no lower than 200 feet above field level (AFL) elevation if the approach 
was to a landing. 

2.2.5. Runway Incursion Prevention System 

Real-time, RIPS algorithms (from NASA/LaRC in-house developments and the Rannoch Corporation) and display 
concepts were integrated with the Synthetic Vision Display Concepts. The G-V aircraft was modified to include an 
ADS-B datalink to support RIPS testing. The existing G-V TCAS was also used for airborne traffic information. 
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2.2.6. SV Sensors 

A modified WxR-2100 commercial, dual-radar receiver/transmitter (R/T) multi-mode weather radar with mounting 
trays, waveguide with a matched load termination, wiring harness, control head, pedestal, and antenna was installed in 
the G-V to support SV Sensor research objectives. The radar antenna was attached to the forward bulkhead of the G-V 
and replaced the current G-V radar and antenna. A wave guide run connected the antenna pedestal and R/T unit. 

During the flight test, the radar operated in one of four modes: 

1 . Weather Radar - standard weather radar functionality 

2. Runway Outline Identification - ground clutter returns were analyzed with aircraft navigational state data to 
provide an estimate runway position 

3. Terrain Feature Extraction (TFE) - ground clutter returns were provided to the DIME as source data 

4. Air-to-Ground Obstacle Detection - radar data processing was used in an attempt to identify objects and 
obstacles on the active runway while on approach 

These modes were manually selected by the SV- Sensors operator depending on the research objective being tested. 

2.2.7. Database Integrity Monitoring Equipment 

A real-time Digital Elevation Model (DEM) integrity monitoring capability has been developed under a Cooperative 
Research Agreement (CRA) with Ohio University. This integrity monitor was designed to detect statistically significant 
differences between sensed terrain features and features derived from the stored DEM, GPS WAAS position 
information, and inertial reference unit (IRU) attitude. Three ranging sensors were assessed. 

1 . The G-V standard radar altimeter - In the presence of unexpected DEM errors, the altimeter-based version of 
DIME provided a real-time loss-of-integrity indication that was used to generate an alert on the Synthetic 
Vision Displays. Alerts were both visual and audible in the flight deck. Radar altimeter-based monitoring is 
described in [11]. 

2. An experimental X-band weather radar - This forward-looking version of the monitor was not integrated with 
the flight deck displays, but was evaluated from an experimenter’s workstation in the aft cabin. This concept 
and preliminary results of post- flight analysis can be found in [12]. 

3. An experimental GPS bi-static radar - This version of the monitor was also not integrated with the flight deck 
displays. Preliminary results of post-flight analysis can be found in [13]. 

2.2.8. Enhanced Vision Sensor 

Enhanced Vision System (EVS) capability was provided by the standard G-V Kollsman FLIR camera. The 
cryogenically-cooled FLIR camera operates in the 1.3 to 4.9 micron wavelength using a sensor with approximately 320 
H x 240 V pixel resolution. The EVS generated an RS-170 video signal which was up-converted to an RS-343 video 
signal for the Flight Dynamics HUD through a scan converter. 

2.3. SVS Software 

2.3.1. Hybrid Terrain Databases 

The SVS terrain employed a hybrid textured format, created by draping monochromatic imagery (aerial photographs) of 
the flight test areas of interest (i.e., RNO and WAL) over elevation-based color-coded digital elevation models (Fig. 2). 
The process produces a false-coloration of the aerial photographs based on altitude above field elevation, thus, 
combining the best benefits of a photo-realistic database (e.g., cultural feature details) with those of a generic-texture 
database (e.g., emphasized terrain elevation). The draped imagery used nesting such that high-resolution imagery (1 
meter per pixel) was used in close proximity to the airfield with the majority of the imagery, away from the airport area, 
at lower resolution (4 meters per pixel). 

The elevation-based color-coding used a green color for the field elevation of the airport changing toward shades of 
brown for higher elevations. For these databases, dark brown represented altitudes closer to field elevation while light 
browns represented higher elevations. At Reno, the shading scheme consisted of 14 elevation color bands that began at 
250 meters elevation and each band was 300 meters in size. At Wallops, the shading scheme consisted of 14 elevation 
color bands that began at sea level with each band representing 10 meters in altitude. 
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Figure 2. Hybrid textured terrain database of RNO area (left picture) and advanced pathway guidance (right picture). 


2.3.2. Advanced Pathway Guidance 

For this flight test, the advanced pathway guidance was comprised of a dynamic tunnel, an integrated cue guidance 
symbol (referred to as the “tadpole”), and path deviation indicators (Fig 2). This pathway guidance combination was 
found to be the optimal based on results from two NASA SVS pathway guidance simulation studies. 5,6 The dynamic 
tunnel concept consists of a series of “crow’s feet” which represented the truncated corners of nominally-connected 2- 
dimensional rectangles spaced at 0.2 nm increments along the desired path. The crow’s feet grow as a function of path 
error to provide the pilots feedback on how well they were flying the defined path. The idea of the dynamic tunnel is to 
minimize clutter when the pilots are flying on path and to alert them as their path error grows by dynamically 
lengthening the sides of the tunnel in the direction of the path error. The tadpole guidance symbol is driven by modified 
pursuit guidance laws and is positioned 30 seconds ahead of the ownship nominally on the centerline of the tunnel. 
Yaw, pitch, and roll attitude of the tadpole reflect the track and flight path angles of the path at that lead position. The 
tail on the tadpole provides the pilot with track change information by rotating left or right in the direction of the track 
change to denote desired lateral path. The path deviation indicators showed “raw data” information (vertical and lateral 
path error) as well as glideslope and localizer deviation (when available) for all the display conditions using error data 
scaled in “dots”. 


2.3.3. HUD Stroke Modifications 

The HUD stroke symbology format was based on the HGS-4000 “Primary Mode” stroke symbology set, albeit with the 
compass rose symbol removed. In addition, a flare cue; glideslope and localizer raw data indicators which included 
deviation scale, angular deviation indication, and path deviation indication (i.e., “dog-bones”); a tadpole guidance 
symbol; and a dynamic tunnel capability were drawn. The HUD tadpole guidance and dynamic tunnel symbology was 
driven identically to that shown on the HDDs. 

2.4. Display Conditions 

Four display conditions (Fig. 3) were evaluated while EPs performed approaches, landings, surface operations, and 
take-offs at RNO and WAL airports as follows: 

• The first display condition was used as our baseline configuration, representative of present-day commercial 
and business aircraft technology, from which the flight test results would be compared. The baseline head- 
down display formats represented a conventional PFD and ND. The PFD contained a flight path marker with 
an integrated guidance cue. The ND was a co-planar display with a map-centered Terrain Awareness and 
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Warning System (TAWS) display and a vertical situation display (VSD). A surface guidance map display was 
presented on the navigation display (when selected by the EP) for scenarios with surface operations. The 
surface guidance map was analogous to present Electronic Flight Bag, Class 3 applications, including ownship 
and other traffic information. The head-up display did not have any raster inputs - it was symbology-only. No 
synthetic terrain information was presented on either the head-up or head-down displays in the Baseline 
condition. 

• The second display condition (hereinafter referred to as Baseline FLIR) had the same head-down PFD and ND 
concepts as the Baseline display condition, but it included FFIR on the raster channel of the HUD. 

• The third display condition (hereinafter referred to as Advanced SVS) utilized the conventional flight 
symbology typically found on a PFD and HUD and also included advanced pathway guidance and SVS terrain 
information. The ND also had SVS terrain information in addition to the TAWS warning and caution overlays 
and VSD. The RIPS display concepts were integrated into the display formats so a surface guidance map 
display (referred to as the electronic moving map or EMM) was automatically presented on the navigation 
display. The display transition strategies were developed in simulation testing prior to the flight test. The 
surface map showed ownship, other traffic, the ATC taxi route and active runways and provided alerting of 
non-normal events (e.g., cross hold-line of active runway, off-route). 

• The fourth display condition (hereinafter referred to as Advanced SVS - No HUD) was exactly the same as the 
Advanced SVS display condition but it did not employ the HUD. Hence, the EPs primary flight reference was 
solely head-down. 

Detailed comparisons of the two NASA Advanced SVS display conditions and the EVS display condition (Baseline 
FFIR) to the Baseline display condition are discussed in Reference 7. 
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Figure 3. Four display concepts evaluated on GVSITE flight test. 
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2.5. Evaluation Tasks 

At each flight test location, EPs flew multiple scenarios which included: approach with wave-off to a departure; 
approach and landing; taxi operations; low-speed rejected take-off; and takeoff and departure. All approaches were 
published instrument or visual arrivals. As possible and practical, emphasis was placed on evaluating the utility of SVS 
displays to enable use of published visual arrivals during (simulated) IMC and night, which are prohibited under current 
equipage; thus, evaluating the potential use of SVS as providing a Virtual-VMC (visual meteorological conditions) 
capability. Waivers were granted to allow the conduct of these visual arrivals at night. 







In addition to nominal approach and departure tasks, there were non-normal runs flown with each display condition 
which included runway incursion (RI) scenarios and database integrity monitoring scenarios. The RI scenarios included 
potential incursions with either a Beech King Air (Be-200) or a specially-equipped recreational vehicle during 
approach, surface, and departure operations. These scenarios were pre-briefed and carefully staged to ensure safety of 
flight and maximize masking of the RI scenario from the EP. The database integrity monitoring scenarios purposefully 
introduced a SV database offset either laterally or vertically with the real world. The pathway guidance was always 
correct and the EPs were instructed to fly with respect to the guidance and not the database image. The EPs were 
instructed to fly each approach as precisely as possible using the display information available to them, as the effect of 
the display information on the EPs ability to fly the approaches would be quantitatively and qualitatively evaluated. In 
addition, the EPs were instructed to taxi as close as possible to the centerline of the taxiway, using a ground speed 
between 15 and 20 knots with a target speed of 18 knots. 

2.6. Organization of Trials 

For this paper, only the nominal approach runs were used in path performance calculations. Non-normal runs, where 
database integrity was intentionally degraded for testing purposes, were only used as appropriate in subjective 
evaluations of specific display elements of interest (e.g., in testing the efficacy of the DIME alerts). 

3. RESULTS 


3.1. Path Control Performance 

There were no significant differences (p>.05) among the four display concepts (Baseline, Baseline FLIR, Advanced 
SVS, Advanced SVS-No HUD) for the measures of RMS lateral path error (mean= 90 ft) and RMS vertical path error 
(74 ft). The pilot performance results are not surprising since each display concept utilized the same pursuit guidance 
control laws and symbology (i.e., the flight path marker, integrated cue guidance symbol, and path deviation indicators). 
These results are also supported by past research. 3,4 The only appreciable difference between the configurations that 
may have influenced path control performance was the addition of the tunnel concepts in the advanced display formats. 
While the presence or absence of tunnel information was not significant in this quantitative path performance analysis, 
it did, as shown in the following, influence the subjective workload and SA measures. The path performance data also 
do not neatly include the influence of guidance and tunnel symbology with off-path starting conditions, because it was 
not possible to precisely control the run-start conditions in the dynamic air traffic/flight test environment; thus, the path 
performance results were normalized by using the tunnel intercept condition (whether the tunnel was explicitly shown 
or not) to begin the path performance “scoring.” 

3.2. Mental Workload and Situation Awareness 

After data collection was completed, each EP completed two separate Subjective Workload Dominance (SWORD) 8 and 
Situation Awareness - Subjective Workload Dominance (SA-SWORD) 9 tests to assess workload and situation 
awareness, respectively: one for display concept (Baseline, Baseline FLIR, Advanced SVS, Advanced SVS - No HUD) 
comparisons during approach and another for display concept comparisons during surface operations. 

SWORD ratings during approach revealed that pilots rated the baseline condition as having significantly higher mental 
workload than the other three display conditions (F(3,33) = 8.470, p < .05). The baseline condition was the only display 
configuration that didn’t explicitly have terrain information on the PFD or HUD, either synthetically or by direct sensor 
display of EVS. For surface operations, there were three unique subsets for SWORD ratings (F(3,30) = 23.196, p < 
.05): (a) Advanced SVS (lowest workload), (2) Advanced SVS - No HUD, and (3) Baseline FLIR and Baseline 
(highest workload). The baseline display concept, again, had the highest mental workload ratings. Two prominent 
display configuration differences influenced the surface operations results - the presence of the Electronic Moving Map 
(EMM) in the advanced display concept and surface guidance symbology on the HUD. The EMM and surface guidance 
symbology on the HUD were significant enhancements to ground operations, thus, reducing the pilot’s mental workload 
when present. 

The SA-SWORD analysis revealed two unique subsets for display concept comparisons for situation awareness (SA) 
during approach (F(3,27) = 8.188, p < .05): (1) Advanced SVS (highest SA) and (2) Advanced SVS - No HUD, 
Baseline FLIR, and Baseline (lowest SA). The Advanced SVS configuration differed from the other three 
configurations, principally by having terrain information on the PFD and HUD. There was also a significant effect 
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found for SA-SWORD for surface operations (F(3,33) = 14.075, p < .05) revealing three unique subsets for display 
concept comparisons: (1) Advanced SVS (highest SA); (2) Advanced SVS - No HUD and Baseline FLIR; and (3) 
Baseline FLIR and Baseline (lowest SA). These results mirror the workload results, signifying the importance of 
advanced guidance and situation information on a HUD for ground operations and improved SA for ground operations 
from the EMM. 

3.3. Subjective Assessments of Information Presentation 

A critical item in the development of an integrated SVS display concept was the determination of when and how the 
display transitions for the HUD, PFD, and ND would occur between the airborne and surface operations concepts. 

These transition strategies were developed in ground simulation using subject matter expert pilots and usability studies. 
In the flight test, using a Likert rating scale (1-7), pilots were asked to rate how effective the Advanced SVS display 
concepts transitioned from: take-off roll to lift-off, approach to landing, landing to rollout, and rollout to taxi. The 
effectiveness scale used went from 1 (completely ineffective) to 7 (completely effective). The average ratings for the 
four transitions was: 5.6 (take-off roll to lift-off), 6.0 (approach to landing), 6.2 (landing to rollout), and 6.2 (rollout to 
taxi). Based on these ratings, the pilots felt that the approach, rollout, and taxi transitions were highly effective; while 
the take-off roll to lift-off transition was effective. Two of the elements of the take-off roll to lift-off transition that 
appeared to earn this lower effectiveness rating was when terrain information was enabled on the HUD and when the 
non-conformal taxi guidance was removed from the HUD and PFD. Pilots commented that the terrain should be 
enabled once the ownship enters the departure runway - not at nose-wheel lift off (the transition point implemented for 
flight test) because during nighttime departure operations, the sudden display of terrain on the HUD (noted by some 
pilots as a “flash”) could be distracting. Pilots also commented that the non-conformal taxi guidance should not go 
away immediately upon entering the runway (the transition point implemented for flight test), but that it should remain 
displayed until the ownship is within +/- 5 degrees of the runway heading. Another common pilot comment was that on 
approach the terrain and tunnel declutter heights (200 feet AFL for the flight test) should be raised to a greater above 
field level altitude. The suggested terrain and tunnel declutter height values ranged from 500-1000 feet AFL as the EPs 
felt that, once stabilized on the approach, their attention shifted away from the approach (and terrain issues) and moved 
toward the landing and runway areas. The terrain information past this stabilized approach height almost becomes 
cluttering and the surface operations display information needs to start coming forth. 

Pilots were also asked to provide two ratings, one on the effectiveness and one on the essentialness, on the presentation 
of NOTAM alerts (e.g., NOTAM tower, closed runway) and DIME alerts for a synthetic vision system. Pilots reported 
that both the NOTAM closed runway alert and the DIME database integrity alert presentation was highly effective and 
essential for a synthetic vision system, but that the NOTAM tower alert presentation was only moderately effective and 
essential. 

3.4. Runway Incursion Prevention Algorithm Performance 

The RIPS algorithm results are very promising, showing successful detection and minimal false alarms. 10 In terms of 
pilot opinion, 9/10 of the pilots reported that the incursion alerts were provided in a timely manner and felt that RIPS 
significantly enhanced RI safety compared to current technology and procedures (cockpit, ground, ATC). After 
familiarization, the majority of the pilots (9/10) trusted the alerting and initiated a go-around or evasive action on the 
ground to avoid a runway incursion. Only one pilot needed to first confirm the hazard before initiating a go-around. 

3.5. DIME Performance 

The performance of the DIME equipment to monitor the SVS database and detect differences between it and the sensed 
terrain data during the flight test is the subject of other publications. 11, 12, 13 While DIME performance is critical to SVS, 
this flight test evaluated, for the first time, a proposed pilot- vehicle interface for indicating a database integrity alert. 

The DIME algorithm uses a test statistic, T, to represent agreement between the stored terrain database information and 
the “sensed” or measured terrain information. Detection thresholds were established a priori based on the published 
quality characteristics of the DEM. During flight, if T exceeded the threshold, an integrity status bit was set indicating 
off-nominal performance. 

Two DIME scenarios were tested: one where the pilot was alerted to the loss of database integrity and one where he was 
not alerted. During “DIME runs”, the terrain database was either: 1) laterally shifted (intentionally) by approximately 
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1000 ft so the SVS presentation of the arriving runway (RNO 16R or RNO 34L) overlaid the position of the parallel 
runway (RN016L or RN034R) out the window or when viewed on the HUD or 2) vertically shifted down by 
approximately 100 feet. The EP’s flight guidance information was unaffected by these intentional database shifts and 
continued to guide the pilot toward the real arriving runway. During an evaluation flight, the EP only saw one type of 
database shift (lateral or vertical) but was exposed to it two times, with and without a DIME “loss of integrity” alert. 

When the database was intentionally shifted, the DIME monitor quickly detected the database shift. When needed 
experimentally, the DIME “loss of integrity” alert was given to the EP as: a) an audible alert to the flight deck; b) a 
“DIME” message written on the PFD, ND, and HUD; and, c) all terrain information on the SVS displays was removed 
(e.g., the PFD went to a “blue-over-brown” attitude director indicator presentation). In general, the “loss-of-integrity” 
was very obvious because of the abrupt display change. The underlying cause of the “loss-of-integrity”, however, was 
not obvious. When the database was shifted, the movement of the terrain on the display was very subtle for the EPs. 
They were not attuned to the database shift and it was not noticed. Several EPs commented that they didn’t like losing 
their terrain display but understood the rationale for its removal (i.e., if the terrain is known to be invalid, it is removed 
since it may represent hazardously misleading information). 

The extent of how subtle the terrain shift was to the EP was most evident when the DIME alert was not used to cue the 
EPs. Most pilots didn’t notice a terrain shift in their displays until very near the runway (>200 ft AFL) if at all. They 
continued to fly the guidance but the synthetically-drawn runway was not “underneath” the guidance symbol. This 
disagreement was not obvious until the synthetically-drawn runway became quite large on the display. One implication 
of this result, found in other studies, is that pilots greatly trust their guidance information. DIME alerts will be essential 
with SVS flight-critical applications and cues or training may be necessary to annunciate or identify significant 
discrepancies that are detected. 

3.6. Voice Recognition and Speech System Performance 

From the data flights, 84% (425 out of 505) of the VRS commands were interpreted correctly. This result was far below 
expectations and experience in other VRS applications. When VRS performance was analyzed on a “per display” 
command (“utterance”) basis, it was found that the ND had a 95% success rate, the HUD had a 96% success rate, but 
the PFD only had a 68% success rate. The voice recognition engine has a known practical success rate of 96% which is 
close to the ND and HUD success rate. Upon further analysis, it was found that a single command was responsible for 
most of the VRS errors. This command was used to change the field of view of the PFD and the command was “P-F-D 
FOV” or “P-F-D Field-of-View”. The VRS was not confident enough in interpretation of this command most of the 
time so “no recognition” was reported by the VRS (i.e., it didn’t exceed the 40% confidence threshold level used in the 
flight test). If the command “PFD FOV” is removed from the data analysis, the overall success rate of the VRS would 
be 96%, which would match our expectations and promote continued evaluation of speech/voice as a natural flight deck 
interface method. 

Phonetic analysis of the commands against the speech recognition algorithm has not been performed but post-test audio 
analysis indicate a more critical VRS performance issue. In general, there was an overall deficiency of audio volume to 
the VRS computer due to set-up/procedural problems during the flight test. Independent of the speech recognition 
algorithm or grammar design, the first principle for the design of a speech recognition system is to have an audio input 
of sufficient volume for the speech recognizer to “hear” the user. 

Overall, the performance of the VRS and the structured vocabulary was very well-accepted. Numerous suggestions for 
improvement were given by each EP - one over-arching theme was that the grammar hierarchy did not include “short- 
cuts.” The pilots wanted to eliminate the need for the first word (the “display”) for all commands which were unique to 
one display or would change all displays. For instance, instead of saying “NAV Range 5”, they wanted to say “Range 
5” since the ND was the only display that used a range control. Many others, such as “Field-of-view”, could have been 
shortened as well. 


4. CONCLUSIONS 

The flight test marked the first time NASA’s technologies have been integrated as a complete system incorporating 
synthetic terrain primary flight and navigation displays, advanced weather radar object detection, synthetic vision 
database integrity monitoring, refined dynamic tunnel and guidance concepts, surface map displays, and the runway 
incursion prevention system (RIPS). The results showed the efficacy of the NASA Synthetic Vision System to 
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significantly enhance pilot situation awareness (without increasing mental workload) for runway traffic and terrain, and 
substantially better pilot acceptability and trust due to integrated integrity monitors and enhanced vision sensors. 

Future research will focus on (1) enhancement of the dynamic tunnel concept to provide 4-D required time of arrival 
and required navigation performance, (2) crew coordination human factors research using SVS, (3) exocentric dynamic 
3-D SVS navigation displays for approach and missed approach rehearsal, (4) military applications of synthetic vision, 
(5) advanced display media, and (6) integration of SVS with other emerging NASA cockpit information displays. 
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